Nonparametric Regression With Basis Selection From Multiple Libraries

New non-parametric regression procedures called BSML (Basis Selection from Multiple Libraries) are proposed in this paper for estimating a complex function by a linear combination of basis functions adaptively selected from multiple libraries. Different classes of basis functions are chosen to model various features of the function, e.g. truncated constants can model change points in the function, while polynomial spline representers may be used to model smooth components. The generalized cross-validation and covariance inflation criteria are used to balance goodness-of-fit and model complexity where the model complexity is estimated adaptively by either the generalized degrees of freedom or covariance penalty. The cross-validation method is also considered for model selection. Spatially adaptive regression and model selection in multivariate non-parametric regression will be used to illustrate the flexibility and efficiency of the BSML procedures. Extensive simulations show that the BSML procedures are more adaptive than some well-known existing non-parametric regression methods. Analyses of real data sets are used to illustrate the BSML procedures. This article has supplementary materials online. Jeffrey Sklar (email: [email protected]) is Associate Professor, Statistics Department, California Polytechnic State University, San Luis Obispo, CA 93407. Junqing Wu (email: [email protected]) is Marketplace Manager, Revenue at Microsoft Advertising, 11155 NE 8th St, Bravern 1/11145, Bellevue, WA 98004. Wendy Meiring (email: [email protected]) is Associate Professor, and Yuedong Wang (email: [email protected]) is Professor, Department of Statistics and Applied Probability, University of California, Santa Barbara, California 93106. Yuedong Wang’s research was supported by a grant from the National Science Foundation (DMS0706886). Address for correspondence: Yuedong Wang, Department of Statistics and Applied Probability, University of California, Santa Barbara, California 93106-3110, USA. 1